The present invention, generally, relates to network communications systems, methods and computer program products and, more particularly, to systems, methods and computer program products for performance testing of computer networks.
Computer networks have grown increasingly complex with the use of distributed client/server applications, mixed platforms and multiple protocols all on a single physical backbone. The control of traffic on the networks is likewise moving from centralized information systems departments to distributed workgroups. The growing utilization of computer networks is not only causing a move to new, high-speed technologies but is at the same time making the operation of computer networks more critical to day-to-day business operations.
The growth in complexity and dependence on computer networks heightens the need for network management tools to design, build and maintain computer networks. The mix of protocols and vendors of installed hardware on many computer networks generally increases the difficulty of accomplishing network management. This problem may arise in planning or designing changes to a network, monitoring performance of a network, and testing the impact on performance of different hardware and software being installed on a network. A variety of approaches to network management tools have been considered including, frame generators, passive monitors, simulation tools, and applications testers. However, each of these categories of network management tools may have its own limitations affecting a users ability to manage increasingly complex and critical computer networks.
Frame generators are typically used for performance and stress testing of network equipment. Typically, they are network analyzers with the ability to send as well as receive data. An example of a frame generator is the DA-30 from Wandel and Goltermann. Another example is SmartBits by Netcom Systems. SmartBits generates packets with fixed- or variable-length gap and packet contents while receiving packets. SmartBits provides for multiple modular ports mounted in a chassis for testing and monitoring internetworking equipment. On the receive side, each card contains counters to monitor the number of packets transmitted, received, collisions, byte count, triggers and certain errors or to monitor the rate of each parameter. A Windows programming interface allows the individual cards to be programmed. Various industry standard bench tests are supported. A third example is LANQuest Net/WRx which enables creation of traffic streams between a sending and receiving node for routers, bridges, hubs, switches and servers acting as routers. Net/WRx provides testing for new and existing network products for interoperability and functional operation. Net/WRx is designed to test product designs for network products. Frame generators are typically specialized, single-purpose hardware which may-be connected to a network to be tested during a test. This can result in high costs, particularly if it is desirable to test a plurality of points requiring multiple systems to be connected for the test. Frame generators are also generally limited in their ability to test network performance under actual application conditions as they typically do not operate on communication nodes of the network to be tested but are instead hardware testers installed to test the network. In addition, they typically operate at layer 2 of the networking model, the link layer. Therefore, typically no real protocol stacks running on the network are used for testing again removing the test from the actual conditions under which the network to be tested would be used.
An alternative type of network management tool is a passive monitor. Passive monitors are a combination of hardware and software that typically monitor network traffic at the link layer. They are passive in that they watch the network traffic that is flowing rather than actively creating traffic. Passive monitors may be categorized as network analyzers, like Network General""s Sniffer, and remote monitors, such as NETscout from Frontier Software Development, Inc., EcoNET from Compuware Corporation, and Vital Signs from VisionNet. However, these tools may also include various limitations. Because they monitor at the link layer, to get a complete picture of performance a monitor is typically required at every network segment, generally causing increased expense and setup requirements. These devices also are limited in that they may only monitor existing traffic as its passes them and may not test by generating predetermined traffic flow patterns.
Another network management tool is a simulation tool. Simulation tools provide a mathematical model of a network. The model is generated based on information about network design, hardware, and traffic patterns. Once the model is created, it can be used to predict network performance under different scenarios. However, these tools are generally limited in their ability to accurately model the network and update the model as the network is changed. In addition, various network problems can arise from subtle differences in configuration, queuing, congestion control and frame sizes which are typically difficult to model. Furthermore, they typically only simulate network performance for when a network is working correctly rather than identifying problems.
A fourth category of network management tools is application testers. These types of systems vary greatly but generally act by driving installed applications on a network. The testers typically simulate a user reading screens and typing at a keyboard to create network traffic. Such programs are generally intended to aid application developers, not network managers. Typically, the application must be available and installed before these tools can be utilized by a network manager. An example of such a test tool is QA Partner from Seque Software, Inc.
Various other testers aimed at addressing particular aspects of network management are available. ProView-SNA utilizes active traffic generation to measure network performance characteristics by monitoring round trip response times to SNA hosts and to LAN file servers. Under control of a ProView Management Station, Remote Agents execute application layer transactions, compute round trip response time and forward the results to the Management Station for display and storage. Net sench is a Ziff-Davis benchmark program that measures the performance of servers in a file server environment. It provides a way to measure, analyze, and predict how a server handles network file I/O requests from client computers in a file server environment.
These network management tools are generally limited in their ability to support multiple protocols and to test networks under actual operating conditions. It is increasingly important to analyze the actual performance of the network to be tested without the constraints and limitations of these existing tools. Additionally, scalabilty of tests, distributed testing and a means to test all types of network devices would provide greater testing flexibility to network managers.
Accordingly, it is an object of the present invention to provide network performance testing systems, methods and computer program products which provide network managers the ability to test the performance of a network in a manner which is representative of actual operating conditions on the network to be tested.
It is a further object of the present invention to provide such network performance systems, methods and computer program products with flexibility in how network performance is measured.
These and other objects are provided by methods, systems and computer program products which define a test scenario including an endpoint node specific test protocol between a first endpoint node on the network to be tested and an associated second endpoint node on the network to be tested. The test scenario may include a plurality of endpoint node pairs each having an associated endpoint node specific test protocol for communications between the pair during execution of the test scenario. The endpoint node specific test protocol is received by one of the endpoint nodes of each endpoint node pair. A corresponding partner endpoint node specific test protocol is determined which defines the communications activities of the associated one of the endpoint node pair to allow the pair to create the desired communications traffic on the network during execution of the test scenario. The partner endpoint node specific test protocol is then provided to the second endpoint node of each endpoint node pair by the first endpoint node of each pair over the network. After both endpoint nodes of the pair have received their respective test protocols they are prepared to generate the desired communication traffic between them when the test scenario is executed. One of the endpoint nodes of each endpoint node pair under the test scenario may monitor performance of the network during execution of its endpoint node specific test protocol.
A console node may be provided on the network to be tested at which the test scenario may be defined based on user input. The console node also provides the first one of each endpoint node pair the respective endpoint node specific test protocol for that pair, initiates execution of the test scenario once each of the endpoint nodes of the endpoint node pairs has its respective test protocol, terminates the test scenario, monitors the performance of the network by receiving performance data reported from an endpoint node of each endpoint node pair executing the test scenario and analyzes the data to generate performance measurements including throughput and transaction rate. Results may be displayed at the console node, saved as a computer file or printed out.
The endpoint node pair based network performance testing of the present invention provides both flexibility and ready scalability for network performance testing. Traffic generation during the test scenario is generated by endpoint node pairs on the network to be tested and the test scenario can provide for varying the number of endpoint node pairs executing protocols under the test scenario as well as varying what endpoint node pairs are involved in the test scenario. Accordingly, physically displaced endpoint nodes can be selected to test devices, such as routers, which may only handle a portion of the communications traffic on a network to be tested. Furthermore, both file servers and user terminals may be designated endpoint nodes for a test scenario to allow user to file server and user to user communications performance to be tested. Regardless of the endpoint node pair combination selected for a given test scenario, the performance data may still be collected and analyzed at the console node to allow for consistent and convenient analysis and reporting of test results regardless of the mix and location of the physical devices actually generating network communications traffic for the test scenario.
The test scenario represents actual operating conditions by operating across various network protocols, such as TCP/IP or APPC, with a plurality of endpoint node pairs on the network to be tested, each executing an endpoint node specific test protocol under the test scenario to emulate communications traffic. Scripts are assigned to each endpoint node specific test protocol to represent a type of traffic such as file transfers, database updates, credit checks, etc. The scripts may be varied to increase or decrease the size and frequency of transactions as well as changing the number of transactions measured per measurement period.
Individual ones of the endpoint nodes of each pair generate communications traffic to the other endpoint node of the pair. Endpoint nodes executing the test scenario may be existing endpoints on the network such as user terminals or file servers and an endpoint node of each pair may measure performance of the network during execution of its respective test protocol. Different endpoint node pairs can be operated with different network protocols and different scripts during execution of the overall test scenario on the network, thereby allowing a varied mix of communications traffic representative of actual network operating conditions. Furthermore, because all of the individual endpoint protocols are associated under the defined test scenario, the test conditions, while flexible enough to represent actual operating conditions, are repeatable thereby allowing testing of changes in network hardware or software under controlled conditions which nonetheless reflect actual operating conditions. Test scenarios may also be varied to stress test different components of the network environment.
Tests may run until each of the endpoint node specific test protocols is completed or until any one endpoint node specific test protocol is completed. Each endpoint node specific test protocol may execute as an application level program driving protocol stacks on the existing hardware of the network to be tested, independent of the actual application programs which may reside on the existing endpoint nodes. Accordingly, no specialized hardware is required because tests may be executed as software objects on existing endpoint node computers on the network to be tested.
Network performance may be described based on performance characteristics such as throughput, transaction rate and response time for the test scenario. Performance measurements may be generated during execution of the test scenario by providing ongoing reporting of performance data from the endpoint node pairs or performance data can be reported after terminating the test scenario to prevent communications traffic associated with reporting from affecting the test results. The flexibility in defining a test scenario and central reporting provide ready access to results and allow for a variety of performance measurements to be generated based upon test objectives of a network manager.
Accordingly, the present invention provides for testing performance of a communications network including a plurality of endpoint nodes under conditions reflecting the actual operating conditions of the network to be tested. A test scenario is defined including an endpoint node specific test protocol between a first endpoint node and an associated second endpoint node to simulate communications traffic therebetween to be tested. A partner endpoint node test protocol is determined from the endpoint node specific test protocol. The partner endpoint node test protocol is communicated over the network from a first endpoint node of an endpoint node pair to the other endpoint node of the pair. The test scenario may also include a network protocol for communications between the nodes of the endpoint node pair thereby providing a mix of network protocols in a single test scenario. Execution of the endpoint node specific test protocols is then initited. Performance of the network is monitored while the endpoint node specific test protocols are executed to obtain performance data. The endpoint node specific test protocols may include a type of data to transmit, how many times to transmit the data, what portion of the test protocol to time during execution, and what acknowledgments to receive during execution of the test protocol.
In another aspect of the present invention, a console node is provided on the network to be tested. The endpoint node specific test protocols may be established at the console node and communicated over the network to be tested from the console node to the endpoint node pairs. The console node may further synchronize initiating of the endpoint node specific test protocols. The capabilities of endpoint nodes may first be detected by the console node before assigning a test protocol and active protocol stacks may also be detected on the endpoint nodes.
In a further embodiment of the present invention, execution of the test protocols may be terminated. The protocols may be terminated and the test concluded for analysis after all of the protocols are completed or after any one test protocol of the test scenario has completed execution.
In a further aspect of the present invention, performance may be monitored at one of the endpoint nodes of each of the endpoint node pairs. The monitored performance data may then be reported to the console node. The performance data may be reported as it is generated during execution of the test scenario or after the test scenario is terminated. Invalid performance data may be removed by the console node from reported performance data.
Accordingly the systems, methods and computer program products of the present invention provide a capability to test networks under actual operating conditions not provided by existing network performance testing tools. Test scenarios are designed around endpoint node pairs to reflect expected operating conditions including the flexibility to provide distinct endpoint specific test protocols to different endpoint node pairs which then execute their individual test protocols under the overall network test scenario. Unlike applications testers, test execution is independent of existing installed applications and provides a test protocol to each endpoint node pair which the endpoint node pair may then execute and monitor. No specialized hardware is required unlike typical frame generators and tests can be readily expanded without a corresponding increase in hardware cost. Unlike passive monitors, the present invention provides for active traffic generation with flexibility to create traffic reflecting actual operating conditions but under repeatable conditions. The present invention thereby provides network managers with an improved tool for designing and managing complex computer networks handling a variety of network protocols, hardware and applications traffic.